admin: add streaming variant of the admin API#32346
admin: add streaming variant of the admin API#32346jmarantz merged 44 commits intoenvoyproxy:mainfrom
Conversation
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
…r tests. Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
|
CC @envoyproxy/coverage-shephards: FYI only for changes made to |
Signed-off-by: Joshua Marantz <jmarantz@google.com>
|
@RyanTheOptimist the coverage change is actually to increase the coverage expectation. I assume that should be an easy one. |
Signed-off-by: Joshua Marantz <jmarantz@google.com>
ravenblackx
left a comment
ravenblackx
left a comment
There was a problem hiding this comment.
Realizing posting a comment just as a comment in a thread doesn't act like a review and doesn't show up in the right place in the conversation timeline so it's easily missed/lost. So this is a request for change connected to this long comment.
Signed-off-by: Joshua Marantz <jmarantz@google.com>
… the lines. Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
|
I got the thread-synchronizer strategy to work but decided it was more verbose and harder to read than the interlock call, so I dropped it. |
Signed-off-by: Joshua Marantz <jmarantz@google.com>
source/exe/README.md
Outdated
| @@ -0,0 +1,10 @@ | |||
| State Action Next State Side Effects | |||
There was a problem hiding this comment.
actually the mermaid diagrams look pretty cool and I will make a note to try them next time, but I propose that what I have is sufficient, and I think it might take a while to get the mermaid diagram to express that cyclic behavior and all the annotations I added to the graphics.
|
Filed #32769 for test flake in ext_proc /retest |
|
/retest |
Signed-off-by: Joshua Marantz <jmarantz@google.com>
|
/retest |
|
@tyxia @ravenblackx are there more comments? Thanks! |
ravenblackx
left a comment
ravenblackx
left a comment
There was a problem hiding this comment.
Thanks for all the fixes.
|
Thansk for the great comments, @ravenblackx and @tyxia . Greg can you do a senior maintainer pass? Thanks! |
ggreenway
left a comment
ggreenway
left a comment
There was a problem hiding this comment.
LGTM. Great documentation and test coverage.
Long ago, we had a discussion about creating an admin http filter in a normal (non-main-thread) http listener. It looks possible that we could use the chunking/buffering mechanism you've built here to implement such a thing. WDYT?
6 participants
Commit Message: In #19693 a streaming stats implementation was introduced, dramatically reducing the amount of time and memory it takes to generate an admin http response. /config_dump (#32054) and /clusters_ (#32054) can also be improved in this way.
However, in some environments we do not want to expose an HTTP port for admin requests, and we must use the C++ API. However that API is not streaming: it buffers the entire content. This PR adds a new streaming API.
Mechanically this new functionality was easy to add as an externally callble C++ API as the Admin class already supported this model on behalf of /stats.
However there's a fair amount of complexity managing potential races between active streaming requests, server exit, and explicit cancellation of an in-progress request. So much of the effort and complexity in this PR is due to making that thread-safe and testing it.
Additional Description:
Note to reviewers: if this PR is too big it can be broken into 2. The significant additions to main_common.cc, main_common.h, and main_common_test.cc need to stay together in one PR, the rest of the changes are non-functional refactors which are needed for the larger changes to work; they could be reviewed and merged first.
Risk Level: medium -- this is a new API but it does refactor some existing functionality. Using the new functionality will add some risk also as there is some careful thought required to believe we have protected against all possibilities of shutdown/cancel/admin-request races.
Testing: //test/..., and lots of tsan repeated tests for the new streaming tests. Lots of iteration to ensure every new line of main_common.cc is hit by coverage tests.
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a
Fixes: #31755